Compression of digital video data is used for many applications including, but not limited to, transmission over bandwidth-constrained channels, such as direct broadcast satellite and storage on optical media. In order to achieve very efficient compression, complex, computationally intensive processes are used for encoding (compressing) and decoding (decompressing) video. For example, though MPEG-2 (Moving Pictures Expert Group, International Organization for Standards, Geneva, Switzerland) is known as a very efficient method for compressing video, a new, more efficient standard, H.264 (“Advanced Video Coding”, International Telecommunication Union Telecommunication Standardization Sector, Geneva, Switzerland), is being developed.
The H.264 standard allows for bitstreams that (i) use a large number of reference frames to reconstruct a single picture and (ii) use reordering schemes that transmit many “future frames” with later display times then a current picture before the current picture is transmitted. By contrast, MPEG-1 and MPEG-2 allow for at most two reference frames for reconstructing a picture and only a single future frame.
Referring to FIG. 1, a diagram illustrating a conventional bitstream 10 that uses many reference frames is shown. In the illustration, a group of pictures (frames) having six pictures is represented by one I-frame 12 followed by five P-frames 14a-14e. The arrows show how reference frames are used for prediction. Each P-frame 14a-14e after the I-frame 12 uses all of the previous frames in the group of pictures as references, so that the last frame 14e has five reference frames 12 and 14a-14d. 
Referring to FIG. 2, a diagram illustrating a bit stream 16 that uses many “future frames” is shown. The bit stream 16 is shown in display order, with arrows showing how reference frames are used for prediction. Like the groups of pictures often used for MPEG-1 or MPEG-2, (i) each P-frame is predicted from one I-frame or one P-frame and (ii) each B-frame is predicted from two frames, each an I-frame or a P-frame. However, the H.264 standard allows the I-frame to be a frame displayed in the middle of the group of pictures. The frames displayed before the I-frame are predicted in the opposite direction as usual; the P-frames before the I-frame use backward only, instead of forward only, prediction. Thus, the I-frame needs to be transmitted first but is displayed tenth. Depending on the order of B-frame transmission (i.e., if the B-frames displayed before the I-frame are transmitted in display order, backwards display order, or something else), the decoder will need to buffer 5-10 frames to decode and display the bitstream.
The flexible approach permitted by the H.264 standard for creating bitstreams results in the decoder buffering a large amount of image data. To limit the amount of memory that a decoder reasonably uses for decoding, the H.264 places two constraints on the bitstreams (i) a bitstream cannot be constructed so that the total number of bytes of decompressed pictures buffered at the decoder exceeds a limit B and (ii) a bitstream cannot be constructed so that the total number of decompressed frames buffered at the decoder exceeds a limit F. For example, for a level 4 (high definition) stream, at any time, the decoder will never hold more than sixteen frames or any number of frames that use more than 12,288×1024 bytes in total.
The H.264 standard allows for a resolution of compressed pictures in a single bitstream to change. Thus, the maximum number of frames buffered over time will vary with the resolution of the frames. When low-resolution frames are used, many frames are commonly buffered. When high-resolution frames are used, few frames are commonly buffered.
If memory is accessed through register read-write instructions using virtual linear memory, the total amount of memory that a decoder will allocate for decoded pictures is roughly the nominal limit given by the H.264 standard (i.e., 12,288 kilobytes (KBytes) for level 4). In practice an actual decoder uses slightly more memory when not operating exactly according to the principles of the H.264 reference decoder. For example, an extra delay between decoding and display introduced for scaling and other display processing consumes additional memory.
A conventional video decoder allocates physically contiguous buffers for the maximum number of frames used for decoding times the maximum size of each frame. The buffers are used for storing frames as the frames are decoded and freed when the frames are no longer needed (i.e., the frames that have been displayed and will no longer be used as references for other frames). The conventional approach will use much more memory than a nominal amount of memory if the number of frames to be buffered depends on the resolution of the frames.